Augmented reality for stroke rehabilitation during COVID-19.

BACKGROUND: The lack of the rehabilitation professionals is a global issue and it is becoming more serious during COVID-19. An Augmented Reality Rehabilitation System (AR Rehab) was developed for virtual training delivery. The virtual training was integrated into the participants' usual care to reduce the human trainers' effort so that the manpower scarcity can be eased. This also resulted in the reduction of the contact rate in pandemics. OBJECTIVE: To investigate the feasibility of the AR Rehab-based virtual training when integrated into the usual care in a real-world pandemic setting, by answering questions of whether the integrated trials can help fulfill the training goal and whether the trials can be delivered when resources are limited because of COVID-19. METHODS: Chronic stroke participants were randomly assigned to either a centre-based group (AR-Centre) or a home-based group (AR-Home) for a trial consisting of 20 sessions delivered in a human-machine integrated intervention. The trial of the AR-Centre was human training intensive with 3/4 of each session delivered by human trainers (PTs/OTs/Assistants) and 1/4 delivered by the virtual trainer (AR Rehab). The trial of the AR-Home was virtual training intensive with 1/4 and 3/4 of each session delivered by human and virtual trainers, respectively. Functional assessments including Fugl-Meyer Assessment for Upper Extremity (FMA-UE) and Lower Extremity (FMA-LE), Functional Ambulation Category (FAC), Berg Balance Scale (BBS), Barthel Index (BI) of Activities of Daily Living (ADL), and Physical Component Summary (SF-12v2 PCS) and Mental Component Summary (SF-12v2 MCS) of the 12-Item Short Form Health Survey (SF-12v2), were conducted before and after the intervention. User experience (UX) using questionnaires were collected after the intervention. Time and human resources required to deliver the human and virtual training, respectively, and the proportion of participants with clinical significant improvement were also used as supplementary measures. RESULTS: There were 129 patients from 10 rehabilitation centres enrolled in the integrated program with 39 of them were selected for investigation. Significant functional improvement in FMA-UE (AR-Centre: p = 0.0022, AR-Home: p = 0.0043), FMA-LE (AR-Centre: p = 0.0007, AR-Home: p = 0.0052), SF-12v2 PCS (AR-Centre: p = 0.027, AR-Home: p = 0.036) were observed in both groups. Significant improvement in balance ability (BBS: p = 0.0438), and mental components (SF-12v2 MCS: p = 0.017) were found in AR-Centre group, while activities of daily living (BI: p = 0.0007) was found in AR-Home group. Contact rate was reduced by 30.75-72.30% within AR-All, 0.00-60.00% within AR-Centre, and 75.00-90.00% within AR-Home. CONCLUSION: The human-machine integrated mode was effective and efficient to reduce the human rehabilitation professionals' effort while fulfilling the training goals. It eased the scarcity of manpower and reduced the contact rate during the pandemics.

Identifying the kind behind SMILES-anatomical therapeutic chemical classification using structure-only representations.

Anatomical Therapeutic Chemical (ATC) classification for compounds/drugs plays an important role in drug development and basic research. However, previous methods depend on interactions extracted from STITCH dataset which may make it depend on lab experiments. We present a pilot study to explore the possibility of conducting the ATC prediction solely based on the molecular structures. The motivation is to eliminate the reliance on the costly lab experiments so that the characteristics of a drug can be pre-assessed for better decision-making and effort-saving before the actual development. To this end, we construct a new benchmark consisting of 4545 compounds which is with larger scale than the one used in previous study. A light-weight prediction model is proposed. The model is with better explainability in the sense that it is consists of a straightforward tokenization that extracts and embeds statistically and physicochemically meaningful tokens, and a deep network backed by a set of pyramid kernels to capture multi-resolution chemical structural characteristics. Its efficacy has been validated in the experiments where it outperforms the state-of-the-art methods by 15.53% in accuracy and by 69.66% in terms of efficiency. We make the benchmark dataset, source code and web server open to ease the reproduction of this study.

Deep Collocative Learning for Immunofixation Electrophoresis Image Analysis.

Immunofixation Electrophoresis (IFE) analysis is of great importance to the diagnosis of Multiple Myeloma, which is among the top-9 cancer killers in the United States, but has rarely been studied in the context of deep learning. Two possible reasons are: 1) the recognition of IFE patterns is dependent on the co-location of bands that forms a binary relation, different from the unary relation (visual features to label) that deep learning is good at modeling; 2) deep classification models may perform with high accuracy for IFE recognition but is not able to provide firm evidence (where the co-location patterns are) for its predictions, rendering difficulty for technicians to validate the results. We propose to address these issues with collocative learning, in which a collocative tensor has been constructed to transform the binary relations into unary relations that are compatible with conventional deep networks, and a location-label-free method that utilizes the Grad-CAM saliency map for evidence backtracking has been proposed for accurate localization. In addition, we have proposed Coached Attention Gates that can regulate the inference of the learning to be more consistent with human logic and thus support the evidence backtracking. The experimental results show that the proposed method has obtained a performance gain over its base model ResNet18 by 741.30% in IoU and also outperformed popular deep networks of DenseNet, CBAM, and Inception-v3.

Contextual Noise Reduction for Domain Adaptive Near-Duplicate Retrieval on Merchandize Images.

In this paper, we have proposed a novel method which utilizes the contextual relationship among visual words for reducing the Quantization errors in near-duplicate image retrieval (NDR). Instead of following the track of conventional NDR techniques which usually search new solutions by borrowing ideas from the text domain, we propose to model the problem back to image domain, which results in a more natural way of solution search. The idea of the proposed method is to construct a context graph that encapsulates the contextual relationship within an image and treat the graph as a pseudo-image, so that classical image filters can be adopted to reduce the mismapped visual words which are contextually inconsistent with others.With these contextual noises reduced, the method provides purified inputs to the subsequent processes in NDR, and improves the overall accuracy. More importantly, the purification further increases the sparsity of the image feature vectors, which thus speeds up the conventional methods by 1662% times and makes NDR practical to online applications on merchandize images where the requirement of response time is critical. The way of considering contextual noise reduction in image domain also makes the problem open to all sophisticated filters. Our study shows the classic anisotropic diffusion filter can be employed to address the cross-domain issue, resulting in the superiority of the method to conventional ones in both effectiveness and efficiency.

Visual Typo Correction by Collocative Optimization: A Case Study on Merchandize Images.

Near-duplicate retrieval (NDR) in merchandize images is of great importance to a lot of online applications on e-Commerce websites. In those applications where the requirement of response time is critical, however, the conventional techniques developed for a general purpose NDR are limited, because expensive post-processing like spatial verification or hashing is usually employed to compromise the quantization errors among the visual words used for the images. In this paper, we argue that most of the errors are introduced because of the quantization process where the visual words are considered individually, which has ignored the contextual relations among words. We propose a "spelling or phrase correction" like process for NDR, which extends the concept of collocations to visual domain for modeling the contextual relations. Binary quadratic programming is used to enforce the contextual consistency of words selected for an image, so that the errors (typos) are eliminated and the quality of the quantization process is improved. The experimental results show that the proposed method can improve the efficiency of NDR by reducing vocabulary size by 1000% times, and under the scenario of merchandize image NDR, the expensive local interest point feature used in conventional approaches can be replaced by color-moment feature, which reduces the time cost by 9202% while maintaining comparable performance to the state-of-the-art methods.

Coaching the exploration and exploitation in active learning for interactive video retrieval.

Conventional active learning approaches for interactive video/image retrieval usually assume the query distribution is unknown, as it is difficult to estimate with only a limited number of labeled instances available. Thus, it is easy to put the system in a dilemma whether to explore the feature space in uncertain areas for a better understanding of the query distribution or to harvest in certain areas for more relevant instances. In this paper, we propose a novel approach called coached active learning that makes the query distribution predictable through training and, therefore, avoids the risk of searching on a completely unknown space. The estimated distribution, which provides a more global view of the feature space, can be used to schedule not only the timing but also the step sizes of the exploration and the exploitation in a principled way. The results of the experiments on a large-scale data set from TRECVID 2005-2009 validate the efficiency and effectiveness of our approach, which demonstrates an encouraging performance when facing domain-shift, outperforms eight conventional active learning methods, and shows superiority to six state-of-the-art interactive video retrieval systems.